Add --runtime-version to functional test Dapr install
#6278
Conversation
Radius functional test overview
Click here to see the list of tools in the current test run
Test Status⌛ Building Radius and pushing container images for functional tests... |
Test Results2 981 tests ±0 2 972 ✔️ ±0 2m 22s ⏱️ +6s Results for commit 8231c63. ± Comparison against base commit 2ca32a6. This pull request removes 2 and adds 2 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
|
For the detailed report, please go to
|
| break | ||
| else | ||
| echo "dapr init failed, retrying..." | ||
| sleep 60 |
There was a problem hiding this comment.
Do we need a minute long wait between retries here?
There was a problem hiding this comment.
This is pretty arbitrary, I'm not sure what the best timeout would be. maybe 30 seconds?
There was a problem hiding this comment.
It's hard to pick an appropriate retry interval without really deeply understanding the failure and its root cause. If the root cause is rate-limiting in the GH API then there's some useful info here:
Another solution would be to install a specific Dapr version. If I understand the history here the failure is querying the github API to determine the version.
There was a problem hiding this comment.
Thanks for the context and sharing the dapr pr link, @rynowak. Looks like we are currently installing a specific version
. From https://github.com/dapr/cli/pull/1134/files and https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limiting, looks like throttling limits are higher if requests are authenticated, so we should do that if we aren't already.@willdavsmith If we aren't sure what the retry interval should be then starting with 60 seconds is pretty aggressive, and exponential backoff would be a better approach. If the problem is indeed rate limiting, then most rate limiting algorithms don't take 60 seconds to replenish tokens. With exponential backoff we can start with a smaller interval and increase it with every retry based on the signal from the server.
There was a problem hiding this comment.
Ummmm acktually .... 🤓
We should start setting --runtime-version ${{ DAPR_VER }}. Our workflow is installing 1.11.0of the CLI and then telling the CLI to install the latest runtime. We need to pass this to the install command here.
If we do this we can hopefully avoid the need for retries.
There was a problem hiding this comment.
@willdavsmith can you please update the pr to include the version flag as @rynowak suggested above?
If we still need retries after adding version, then let's start with a smaller interval and exponentially backoff as I mentioned above.
Radius functional test overview
Click here to see the list of tools in the current test run
Test Status⌛ Building Radius and pushing container images for functional tests... |
|
For the detailed report, please go to
|
Radius functional test overview
Click here to see the list of tools in the current test run
Test Status⌛ Building Radius and pushing container images for functional tests... |
|
For the detailed report, please go to
|
willdavsmith
force-pushed
the
willdavsmith/dapr-retry
branch
from
a4f48db to
c7df4f2
Compare
willdavsmith
changed the title
--runtime-version to functional test Dapr install
Radius functional test overview
Click here to see the list of tools in the current test run
Test Status⌛ Building Radius and pushing container images for functional tests... |
Radius functional test overview
Click here to see the list of tools in the current test run
Test Status⌛ Building Radius and pushing container images for functional tests... |
|
For the detailed report, please go to
|
Radius functional test overview
Click here to see the list of tools in the current test run
Test Status⌛ Building Radius and pushing container images for functional tests... |
|
For the detailed report, please go to
|
Description
This might help with the increase in functional test failures we've been seeing.
Type of change
Fixes: #6213
Auto-generated summary
🤖 Generated by Copilot at 5eb2160
Summary
🔁⚡🐛
Retry
dapr initin functional test workflow. This improves the reliability of the test suite for the radius project, which depends on dapr.Walkthrough
dapr initcommand to handle transient errors (link)